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ABSTRACT 

The Microcomputer Diagnostic Testing Project (MDTP) 
was designed to broaden the use of microcomputers in student testing, 
for elementary and secondary school use, and for a variety of test 
item formats. MDTP was designed to administer tests, to allow 
teachers to create or revise tests, to print out test forms when 
desired, to score tests* to perform item analysis, and to be 
user-friendly. The first year of the three-year project resulted in 
the development of multiple choice, true-false, and A or B test 
items. System programs were written in BASIC for Apple 
microcomputers. Field testing classrooms were selected from several 
school districts in North-Central Alberta, Canada. Feedback was 
obtained and modifications were made as the project continued. There 
were problems associated with the way computers were distribued in 
schools, and the way in which various types of test formats were 
administered. It was found that this testing program was more readily 
used at the elementary level. It was also discovered that a Sample 
way to incorporate mathematical characters was needed. Teachers' 
attitudes, based on preliminary surveys, were positive, but not 
overwhelmingly positive. (GDC) 
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ABSTRACT 

Th* bulk o-f microcomputer testing 1 i t*r*tur* has b*en concen- 
trAttd in item banking And computer adaptive testing systems. 
Common to both formats is use of a multiple choice format and a 
bias towards upper-secondary and college age students. A more 
general model of testing is presented to help broaden the use of 
microcomputers for testing. Programs are being developed and 
examined for each of the components in a school setting. Before 
microcomputers can be widely accepted in schools problems with 
logistics, flexibility of program format, flexibility of testing 
format, and independence of testing components must be overcome. 
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A G«n*ra1 Testing Mod«1 -for Mi crocomputtr Assessmffnt in Education 

As th« number of microcomputers increases throughout educ«i*- 
tion, so do the different uses to which the microcomputer is put. 
Among its facilities, the mi crocomputf r offers the potential for 
individualized testing, automatic scoring, and randomized ques- 
tion ordering. However, if it is the desire of education to have 
microcomputers to take on this new role for the long run, then 
care must be taken in developing a model that is properly suited 
to the needs of education. 

This paper presents a general model for assessment in educa- 
tion using microcomputers and discusses the initial microcomputer 
programs designed to U3e the model. We have developed the model 
into the iii crocoinpu ter Diagnostic Testing Project (MDTP> . The 
goal of the model associated with MDTP is to provide a workable 
framework for the development of a series of microcomputer pro- 
grams that present different types of test formats, such as 
multiple-choice, f a 11 -i n-the-bl ank , and oral presentation for- 
mats. In addition, MDTP allows the teacher to use as many or as 
few of the different components as desired. Finally, this pro- 
ject attempts to demonstrate the way in which such a system can 
gain widespread acceptance throughout different grade levels and 
subject areas. 

Several interesting trends emerge from an examination of the 
literature on microcomputers and testing. The bulk of the liter- 
ature seems to be concentrated in two specific areas. The firnt 
area is Tn developing item banking programs where the user has a 
choice of questions for each test item from a larger pool of 
items <cf. Hsu ic Nitko, 1984{ Johnson, Uillis, Seely, it Moore, 
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1981 >• This Allows stv«rAl -forms of tht same test to given, 
minimizing the risk of cheating «nd nl lowing detesting on the 
sune miiteri«i1.| Item bank programs are typical, y used with col- 
lege or upper secondary level students where the demand on the 
instructor is often quite high due to the large number of stu- 
dents that will take a test at the same time and the repeti- 
tiveness of the material being taught over time. The second area 
of research concentration is computer adaptive testing systems 
<cf. McKinYey it Reckase, 1980{ Uale, 1981). Ui th this form of 
testing, the goal is to accurately assess the area being measured 
with fewer items and in less time. Typically, with computer 
adaptive tests, each item is chosen based on the studenfs per- 
formance on previous items. Most of the computer adaptive test- 
ing research is limited to college age students «rd in the mili- 
tary where aptitude tests are of high importance. Because much 
of the testing research has been concentrated to item banking and 
computer adaptive testing, most of the research has been con- 
ducted with college level and upper secondary students. 

R common factor of these and other microcomputer testing 
programs is the multiple choice test format. A great deal of 
literature exists on statistical methods for testing item valid- 
ity and reliability within the multiple choic* testing format 
(cf. Thorndike, 1971); the computer is well suited to perform 
these operations. This analytic ability, plus the simple use of 
the computer's keyb -^d makes microcomputers well suited for this 
type of testing. 

This dominance o-? the multiple choice format may not cor- 
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respond to th« frequtncy o-f tht gontral -formiit in •ducat ion *s a 
Mho1», howfv*r| •specially at the •Icmentar/ level or non-science 
Humect areas where subjects such as spelling (c-f. V^arnhagen, 
1984) are tested. Uhile computers are well suited to the needs 
of item banking and adaptive testing, the needs o-f education 
itself across grade levels and subject areas should be carefully 
examined so microcomputers can best be used. For example, if the 
short answer testing format is found to be heavily used at the 
elementary school level, computer testing programs should be 
written accordingly. 

The MDTP model was originally developed to demonstrate the 
following functionsi 

I> Catalog and briefly describe the tests that are included in 
the system. 

A> Catalog and describe other tests that are not part of 
the system (including written, non-compitter tests). 

II> Create new tests for the system or revise existing items by 
the teacher or other users. This feature allows the 
entry/revision of new tests or tests previously given in 
written form. 

Ill) Administer the test on computer to students on an indivi- 
dual basis. Students are able to take any test previously 
entered on the MDTF system. 

IKJ^ Print out tests entered into the system, included tests to 
be printed out for written administration only. Uith this 
option system tests can printed out and given to the class 
in written form) for example, when facilities do not permit 
individualized computer testing, 

Kf> Store the results for later retrieval either after a stu- 
dent has taken the test on the computer or if added separ- 
ately by either the teacher or from a card reader. 

KJl> An option for reading mark-sense paper and then treating 



the data in the same manner as data from tests administered 
on computer. 
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VII) Reliably score the results of tests from preset criterion 
und perform « limited interpretation of the data based on 
preset criteria previously entered by the teacher (e.g., 
9Z/i or hiQher - "A", 8i2J^ - 9VA - -B' , etc). 

VIII) Print out various types of reports based on the data, 

i ncl udi ngi 

A) Student scores by cfass list. 

B) I tern analysis of the different questions. 

IX) Maintain system flexibility so that the programs can be 
continually expanded to incorporate addi'^ional features or 
modified for specific applications. 

A) For example, the ability to drive a cassette recorder 
would alloM the system to administer prerecorded oral 
spelling tests or give other types of verbal tests and 
instruction. 

X) Finally, the system attempts to be simple and sufficiently 
standardized to be used effectively by cc«nputer*-ha i ve 
teachers an^ students. This includes having consistent 
conventions throughout the different programs. 

The MDTP system has recently been implemented in the 
schools. It is continually being evaluated by the teachers using 
the system. Modifications are made to the programs and to the 
general model based on these comments and suggestions. Our goal 
is to develop a system that teachers will use and find easy to 
incorporate into their classroom testing. The specific benefit 
to the teacher and student is continually being examined. In 
addition, care is being taken to not have the computer used in 
situations where existing rethods are easier or more beneficial. 
Finally, the effect on the students of the system is being close- 
ly examined. Ue have not assumed that the student will necessar- 
ily perform the same on or off computers. 

The first year (April 1984-August )1985) of the three year 
project has involved developing specific components of the pro- 
grams and finalizing the model itself. Currently, forced-choice 
(i.e., multiple choice, true-false, and A or B) tests are the 
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only testing forin«t operating. Progriims «r* written in BASIC for 
Apple microcomputers. 

Field test clf.sses were «e1&ct;u om several school dis- 
tricts in Nor th~centra1 Alberta, Canada. The classes include 
different grade levels ?!nd subject areas, including French immer- 
sion. Teachers agreed to use the programs in conjunction with 
normal testing in their classroom. The teachers were instructed 
to attempt to identify ti^stlng applications that are appropriate 
for the programs, but not to force their tests to fit the re- 
quired testing format f 

During this first year of the project, most of the time was 
spent developing the system components, - obtai nj ng feedback from 
teachers who use the system, and field testing the programs. 
Emphasis was placed on the creation of the test development 
programs themselves, with program additions and modifications 
made in response to teacher suggestions. These continual modifi- 
cations were made at the temporary cost of being able to develop 
all the necessary programs for the model. The components that 
have been delayed include the test description, limited inter- 
pretation, and item analysis components of the MDTP model. 

Although the model has provided an adequate general frame- 
work for the development of the system programs, several specific 
requirements have emerged and are important to take into account 
in the development of testing programs. In addition, some of the 
problems that have been encountered during the project may be 
potentially found in other microcomputer testing related situa- 
tions. A fundamental difficulty concerns the way computers are 
distributed in many schools. Either computers Are distributed 
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ncross th9 entir* school with, «t best, two computers per class- 
room or the computers are concentrated in one or two locations in 
frfi crocompu ter labs. If computers are distributed across the 
school, then the testing must be done individually and across a 
long period of time. Th.i s often, is in contrast to the way tests 
are typically given, fn which the entire class takes & test at a 
specific time. On the other hand, if the computers are concen- 
trated in a lab setting, then testing can be accomplished in one 
or two si tt i ngs; however , enough copies of the program, and the 
computer-administered test must be available for each machine. 
There is no guarantee that the computer distribution in a partic- 
ular school will match the way in which the teachers in that 
school g«'nera11x administer their tests. 

Related to this fundamental problem, ^t was found that a 
paradox exists betwec^n the type of tests and the wax in which the 
tests are administered. Elementary classes are probably best ^ 
suited to individualized presentation of tests but, on the whole, 
elementary school teachers ui^e mustiple choice tests the least. 
On the other hand, multiple choice tests increase in importance 
throughout secondary education as class size increases, but in- 
dividual ization becomes increasingly impractical due, among other 
factors, to the increased need to test an entire class at one 
time. It then becomes more efficient to use this system simply 
as a means to write the tests, have the tests printed out, given 
in the normal paper and pencil way, and then hand scored. In the 
future, when the card reading facility is available, the results 
can be read back into the computer. 

> 
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. ^ Rvgardl so fur we hAvt found that our progrwn is^mort 

readily used at tht tlementar/ legtl. Thi» i« HKtl/ due .to the 
following rea&onfti ^irmt, I ogl st i cal 1 y , computers and classes are 
often suited to individualized assissment in the elementary 
scho6) setting. ThAt is, •1«m«nt«iry classes ure often broken 
into various groupings that work on the same subjects «t dif- 
ferent paces, so testing mA^ Occur in small groups or on an 
indi u i dual i zed basjs. Second, elementary teachers as a rule 
perform more i nf ormal assessments of their classes. That is, 
elementary teachers are generally likely to give many-^short 
quizzes in the various subject areas taught. Finallyp elementary 
tests are often simpler and shorter than those required ihjt 
secondary level. That is, both the length of the test questions, 
and the number of questions on a typica? test will generally be 
shorter at the el^m'rntary school level. It had not beea our 
original intent to develop our system to favor any level of 
education, but due to these major factors and others, our system 
has initially been found to be best suited for testing at the 
el ementary 1 evel . ^1 

Anotl^er problem is being able to dear wiyCh special charac- 
tei^s (such as the division sign), illustrations, and -figures. It 
is possible to modify a computer program to allow for special 
characters and figures, but the relative simplicity of the pro- 
gram is then sacrificed. This is the case because it is neces- 
sary for the user to learn additional procedures to choose the 
appropriate character sets in addition to the other standard 
question entry procedure'^. Still, because of the large demand 
for mathematical characters, a current programming priority is to 
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^IIgw -for th# Addition of special ch^racttri. 

Th« data sources used to evaluate the program and the model 
come from the ability to write and modify the programs according 
to explicit objectives and continual formative and summative 
evaluations performed in the classroom. In addition, teachers 
and students are being given surveys for 'their conments and 
opinions One of the major means of evaluating the system con- 

i 

ctrns the ttachers^ intertst in continued work with the progrvnft 
after the initial evaluations and interest of other teachers to 
become involved in the project. ^ 

To this pointy there has been continued teacher interest in 
the program. Data is^eing continual 1x collected as the field 
tests proceed. Verx preliminary teacher survey^ data indicates a 
positive but not overwhelming, positive attitude toward the MOTP 
programs. The teachers see the s/stem as comparable to thf* 
procedure for creating a written test in terms of the amount of 
time required. The/ also express an interest in continued use of 
the programs, especially if changes they have suggested a. ^ made 
and if the program remains available for an entire semester. The 
teachers felt they were at a particular disadvantage In trying to 
use this system in the middle of the semestf:r. 

Evaluation data is continually being collected and modifica- 
tions made accordingly. This interaction between teachers and 
the program developers has led to the development of important 
features that allow greater generality of the MOTP system. One 
of the original suggestions made by a teacher examining the 
program was to include option to allow students to skip the 
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quefttion bting pr*ttnt*d. After 4l 1 the questions were presented 
they would be re-presented for review and the student would have 
the option of chunQing his or her answer. Since this is possible 
with most written tests, it was decided to incorporate this 
option, in the MOTP system as the skip/review optionu 

Uhen skip/review is used, the student is presented with the 
choice of skipping the question in addition to the different 
answer options. After the initial presentation of the questions, 
each question is shown a second time. The student has the choice 
of doing nothing, accepting his or her answer, or changing the 
:insw(«r. At the end of this and subsequent presentations, the 
student] is asked if he or she wants to review the questions and 
answers yet another time. 

In order to examine whether and how the feature is used by 
the student, MDTP records on disk the student's answers froir^ both 
before and after the the review procedure, keeping track of items 
the student skipped and changes the student made to his or her 
answers. Through comparing the files, the use of the skip ption 
and the ability to change answers can be easily noted. 

Preliminary data has shown that about half of the students 
have not taken advantage of this, feature; in other words, they 
have not used the skip option or changed their answer during 
review. For those students who ^ use the feature, their overall 
test ^core tends to either remain the same or improve through 
using the skip option. Uhen a student skips a problem, he or she 
tends to subsequently get the proolem correct about SO percent of 
tL time, which is above a chance level. 

If this finding is upheld with subsequent data, then the 
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importanct of allowing skipping and r*vl«uiing will b* supported. 
In additjon, thts* results would imply that non-inclusion of this 
feature in a computer i ztd multiple choice program may hurt the 
scores of some students. Given this knowledge, it may be possi- 
ble to challenge the comparability of other computerized testing 
programs and a normal paper and pencil test where skipping and 
review can be a matter of course. 

The awareness of the potential aid that microcomputers can 
offer to student assessment is becoming increasingly better un- 
derstood. Programs now exist that allow for item banking, adap- 
tive testing, creation oi' simple multiple choice tests, and aid 
in analysis and recordii of results. However, currently most of 
the systems are separate and disjoint. Those systems that are 
more general often have several restrictions in format and cannot 
incorporate deviations. The MDTP system, on the other hand, is 
flexible to the needs of teacher, can be us(^d in a component 
manner or as an entire system, and is employed in several grades 
and subject matters. It should be relatively simple to modify 
the existing programs in response to feedback we have received 
from our field testing experience and surveys. For example, we 
hope to learn which features were most important and useful to 
the teachers so we know where to concentrate our own effort. 
This information in turn is helping us better understand the 
testing process itself. Ue hope to continue to develop MDTP 
according to our model, and, if it gains widespread acceptabil- 
ity, then we will have demonstrated an effective use of micro- 
computers in education. 
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